Dealing with Uncertainties in User Feedback: Strategies Between Denying and Accepting

Latest research revealed a considerable lack of reliability within user feedback and discussed striking impacts for the assessment of adaptive web systems and content personalisation approaches, e.g. ranking errors, systematic biases to accuracy metrics as well as its natural o‚set (the magic barrier). In order to perform holistic assessments and to improve web systems, a variety of strategies have been proposed to deal with this so-called human uncertainty. In this contribution we discuss the most relevant strategies to handle uncertain feedback and demonstrate that these approaches are more or less ine‚ective to ful€l their objectives. In doing so, we consider human uncertainty within a purely probabilistic framework and utilise hypothesis testing as well as a generalisation of the magic barrier to compare the e‚ects of recently proposed algorithms. On this basis we recommend a novel strategy of acceptance which turns away from mere €ltering and discuss potential bene€ts for the community of the WWW.